Adverse Leader Selection & Evaluation

Leadership Matters



  • Critical to the success of individuals, teams, organizations, and society at large
  • Strong incentives to place the right person in charge
  • Despite this importance and the strong incentives at play…



We fail to select the best leaders

  • 50 to 75% of selected leaders fail

Adverse Leader Selection





  • The wrong people make it to the top; the right people do not
    • Leader over-emergence & leader under-emergence
  • Leader emergence vs. leader effectiveness is…
    • Higher: Leader over-emergence
    • Lower: Leader under-emergence

Costs to Leadership Failure



  • Financial costs
    • $750,000 to 5 million per failed executive
  • Psychological costs
    • 75% of working adults: Boss is most stressful aspect of the job
    • 70% of US workers: Would pay to have their boss fired
    • Bosses are often the primary reasons employees quit
  • Societal costs
    • Leadership scandals & ethical lapses
    • Military disasters

Why Are We So Bad At This?!



Why the wrong people make it to the top, and the right people do not



Trait lens

  • Ambition

Behavior lens

  • (Un)certainty displays

Process lens

  • Opt-in selection mechanisms

Trait

Ambition ➙ Leader Emergence



  • Ambition is “a persistent and generalized striving for success, attainment, and accomplishment”

  • Ambitious individuals are more likely to emerge as leaders

    • Higher levels of education
    • Higher GPA
    • More prestigious occupations
    • Higher ranking positions
    • Higher salaries
  • But are ambitious individuals also more effective in leadership roles?

Ambition ➙ Leader Effectiveness?



Self-awareness: Individuals struggle to gauge their own effectiveness

Self-serving biases: Individuals often have inflated self-perceptions

Motivated reasoning: Allure of incentives





H1: Ambition is positively associated with self-rated leader effectiveness

H2: Ambition is not associated with other-rated leader effectiveness

Is Ambition Well-Calibrated?

  • Study 1A & 1B: 360-degree assessments of executives
    • N ratings = 3,813
    • N leaders = 468
    • Raters = Self, peers, subordinates, and managers
  • Study 2: Leadership competition among MBA students
    • N ratings = 829
    • N leaders = 181
    • Raters = Expert judges
  • Study 3: Leadership task with random assignment of a nationally representative sample
    • N ratings = 305
    • N leaders = 101
    • Raters = Group members

Analytic Approach

Null Hypothesis: No relationship between ambition and third-party ratings

Alternative Hypothesis: A relationship between ambition and third-party ratings

  • Can we reject the null hypothesis?
  • p < .05: Reject the null hypothesis
    • Interpreted as evidence for the alternative
  • p > .05 : Inconclusive
    • Fail to reject” the null
    • Cannot be used to accept the null
  • Can we reject the presence of a meaningful effect?
  • p < .05: Reject the presence of a meaningful effect
    • Conclude statistical equivalence
    • Declare absence of a meaningful effect
  • p > .05: Inconclusive
    • Fail to reject” the presence of a meaningful effect
    • Cannot be used to declare presence of a meaningful effect
  • Can we accept the null hypothesis?
  • Bayes factors: How much do the data support the null hypothesis?
  • \(BF_{0} > 1\): support for the null hypothesis over the alternative
    • Accept the null, reject the alternative
  • \(BF_{0} < 1\): support for the alternative hypothesis over the null
    • Accept the alternative, reject the null

Study 1A & 1B

  • 360-degree leadership assessments completed by executives’ managers, peers, and subordinates
    Show details
    • The proprietary 360-degree assessment includes 50 items assessing
      10 leadership competencies (e.g., motivation, communication)
    • Two sample items for the motivation competency are
      “Motivates others to put forth greater effort” and
      “Recognizes others’ contributions”
  • Executives reported a self-assessment and their ambition
    Show items
    • My ultimate career aspiration is to be in a position of senior leadership
    • I have strong ambition when it comes to my career goals
    • I am highly motivated to get promoted quickly and often
    • I want my job to be highly respected by others
    • I am determined to have a highly successful career
    • It’s important to me to attain a high-status position in my career
    • I do NOT have a strong desire to advance far in my career (R)
  • Study 1A: N = 1,371 ratings
    • n = 177 self-ratings
    • n = 1,194 other-ratings
  • Study 1B: N = 2,442 ratings
    • n = 291 self-ratings
    • n = 2,151 other-ratings

Study 1: Results

Positive relationship between ambition and self-ratings of leadership effectiveness

  • Study 1A: b = 0.09, 95% CI [0.03, 0.16], t(175.04) = 2.85, p = .005
  • Study 1B: b = 0.08, 95% CI [0.03, 0.13], t(288.95) = 3.05, p = .003

Null relationship between ambition and other-ratings of leadership effectiveness

  • Study 1A: b = 0.04, 95% CI [\(-0.01\), \(0.08\)], t(172.57) = 1.46, p = .146
  • Study 1B: b = 0.01, 95% CI [\(-0.03\), \(0.04\)], t(289.02) = 0.41, p = .680
  • Third-party ratings of leader effectiveness are statistically equivalent, regardless of ambition
    Show details
    • 1A: Test against \(\beta \pm .20\), t(172.57) = \(-6.82\), p < .001
    • 1B: Test against \(\beta \pm .20\), t(289.02) = \(-10.26\), p < .001
  • Bayes Factors show support for the null hypothesis
    Show details
    • 1A: \(BF_{01}\) = 6.29
    • 1B: \(BF_{01}\) = 20.40
    • The data are 6 to 20 times as likely under a null model

Study 2

  • Expert-evaluated leadership skills competition
    Show items
    • This person has all the qualities that would make them an effective leader
    • This person would succeed as a leader in my company
  • N = 829 ratings of 181 MBA students

H2: Null relationship between ambition and other-ratings of leadership effectiveness

  • b = \(-0.06\), 95% CI [\(-0.16\), \(0.04\)], t(175.09) = \(-1.18\), p = .241
  • Third-party ratings of leader effectiveness are statistically equivalent, regardless of ambition
    Show details
    • Significant test against \(\beta =.20\), t(175.09) = \(-5.04\), p < .001
    • Significant test against \(\beta = -.20\), t(175.09) = \(2.69\), p = .004
  • Bayes Factors show support for the null hypothesis
    Show details
    • \(BF_{01}\) = 3.64
    • The data are over 3 times as likely under a null model

Study 3

  • Group decision-making task with randomly assigned leaders vs. followers
  • Nationally representative sample: N = 305 ratings of 101 group leaders
    Show items
    • I would want this person to be my boss or supervisor at work
    • This person has clear leadership potential
    • Overall, this person was NOT an effective leader (R)
    • If I had the chance, I would definitely want to have this person as my leader again
    • Looking ahead, I expect this person will experience great success as a leader

H2: Null relationship between ambition and other-ratings of leadership effectiveness

  • b = \(-0.06\), 95% CI [\(-0.18\), \(0.06\)], t(98.99) = \(-1.02\), p = .312
  • Third-party ratings of leader effectiveness are statistically equivalent, regardless of ambition
    Show details
    • Significant test against \(\beta = .20\), t(98.99) = \(-4.36\), p < .001
    • Significant test against \(\beta =-.20\), t(98.99) = \(2.33\), p = .011
  • Bayes Factors show support for the null hypothesis
    Show details
    • \(BF_{01}\) = 3.98
    • The data are about 4 times as likely under a null model

Uncalibrated Ambition

Ambitious individuals are judged as no more effective in leadership roles

  • Executives rated by peers, subordinates, and managers in a 360-degree assessment
  • MBA students judged by experts in a leadership skills competition
  • A nationally representative sample of U.S. adults leading a small team in a leadership task

However…

  • Ambitious individuals are more likely to emerge as leaders
  • Ambitious individuals report more positive views of their leadership effectiveness

Emergence-Effectiveness Gap

  • Ambition is uncalibrated to actual effectiveness
  • Ambitious individuals over-emerge as leaders

Why Does This Matter?

  • 50 to 75 percent of selected leaders fail
  • Leader selection mechanism is often “opt-in”
    • Self-selection into candidate pool
    • Assumption: those with the greatest leadership potential will apply
    • We show that this assumption may be unfounded
  • We may need an alternative approach to leader selection and development
    • How can we select the best future leaders?

Behavior

Does Expressing Uncertainty Help or Harm Leaders?

Leader-Expressed Uncertainty



“Society tends to attribute authority to those who convey certainty
rather than to those who emphasize uncertainty



Past research outlines two opposing positions:

  • Expressing uncertainty is detrimental to leaders’ social influence
  • Expressing uncertainty is beneficial to leaders’ social influence

How can we reconcile these two perspectives?

Social Influence



“The ability of one person to influence the attitudes and behavior of another”

Uncertainty May Harm Leader Influence



Competence-signaling account

  • Expressing confidence is effective across a wide range of social settings
    • Courtroom, military, political arena, hiring committees…
  • Advice-giving
    • Confident advice is more likely to be followed, regardless of accuracy/quality
    • Physicians, brokers, consultants, therapists, peer reviewers…
  • Presumption of calibration
    • A person’s expressed confidence corresponds to their actual competence

Uncertainty May Harm Leader Influence



Leadership-prototypes account

  • Implicit assumptions about the traits, abilities, and behaviors that characterize an ideal leader
    • “Implicit leadership theories” or “leadership prototypes”
  • Humans hold highly romanticized, heroic views of leadership
    • Charisma
    • Dominance
    • Assertiveness
    • Expertise
    • Confidence

Uncertainty May Harm Leader Influence



Taken together, the competence-signaling and leadership-prototypes accounts suggest…













Hypothesis 1: Leader-expressed uncertainty has a negative effect on leader social influence

  • Perceiver attitudes (H1a)
  • Perceiver behaviors (H1b)

Uncertainty May Benefit Leader Influence



Humble leadership

  • Showing “humanness”: admitting limitations and knowledge gaps

Authentic leadership

  • Knowing and acting upon true values, beliefs, integrity, and strengths
  • Involves self-awareness: authentic leaders are conscious of knowledge gaps

Vulnerability and self-disclosure

  • Disclosing personal limitations and weaknesses

Positive outcomes of humility, authenticity, vulnerability, and self-disclosure include…

  • Trust, liking, engagement, psychological safety, feedback seeking, knowledge and information sharing, job satisfaction, job and task performance, creativity, organizational citizenship behavior, job retention, team learning

Uncertainty May Benefit Leader Influence



Taken together, the literatures on humility, authenticity, vulnerability, and self-disclosure suggest…













Hypothesis 2: Leader-expressed uncertainty has a positive effect on leader social influence

  • Perceiver attitudes (H2a)
  • Perceiver behaviors (H2b)

Help, Harm, or… Both?



  • Literature may be reconciled by examining nonlinear effects
  • Curvilinear effects have been theorized, but most empirical studies rely on binary measures of (over)confidence
  • Too-much-of-a-good thing” effects are pervasive in leadership research
    • Seemingly positive relations that become flat or negative after reaching a specific inflection point
      Show examples of TMGT effects Agency, communion, decision latitude, empathy, empowering leadership, ethical leadership, intelligence, optimism, participative leadership, psychopathy, tenure, unpleasantness, voice, warmth
    • Assertiveness & charisma: involve confident communication
  • Perils of overconfidence; observers prefer calibration in confidence and communication
    • Expressing uncertainty when appropriate
    • Job applicants, collaborators, advisors, eyewitnesses, managers…

Costs of Miscalibrated Confidence



Miscalibration of confidence harms…

  • Perceived credibility
  • Perceived value to the team
  • Perceived desirability
  • Trust
  • Willingness to collaborate
  • Willingness to follow

Help, Harm, or… Both?



Taken together, too-much-of-a-good thing effects, perils of overconfidence, and preferences for calibration suggest…













Hypothesis 3: Leader-expressed uncertainty has a curvilinear (inverse U-shaped) effect on leader social influence

  • Perceiver attitudes (H3a)
  • Perceiver behaviors (H3b)

Meta-Analysis



  • Studies that manipulate or measure expressed uncertainty
    • Include a zero-order effect for an interpersonal outcome
    • Minimize common-method bias (split-samples/time lags)
  • Obtained correlation in support of H1/H2
    • Recoded to express effect of uncertainty
  • Random-effects meta-analysis
    • k = 20 studies
    • m = 87 effect sizes
    • N = 28,594 subjects

Meta-Analysis

Meta-Analysis: H1

Meta-Analysis: H2

Meta-Analysis: Takeaways



Meta-analytic estimate of the interpersonal effect of expressed uncertainty

  • H1 (negative): r = \(-\).26, 95% CI [\(-\).30, \(-\).23]
  • H2 (positive): r = .13, 95% CI [.07, .20]



We suggest a non-linear test (H3)
to reconcile these conflicting results

Methods

  • Five between-subjects experiments
    • S1: Decoding game
    • S2: Emotion estimation game
    • S3: Anagram game
    • S4: Signaling game
    • S5: Prediction game
  • Experimentally manipulate leader-expressed uncertainty
  • Three core attitudinal outcomes
    • Leader effectiveness, warmth, competence
  • Three core behavioral outcomes
    • Advice-following/performance, leader selection, leader reward
  • Incentive-compatible
    • Prespecified bonus for each correct answer

Manipulating Uncertainty



  • On a 9-point scale ranging from highly certain (-4) to highly uncertain (+4)
  • Three types of uncertainty expressions
    • Calibrated scale (“On a 1-9 scale, where 1 = highly uncertain and 9 = highly certain, my certainty level is a 7”)
    • Percentages (“I am 80% sure”)
    • Colloquial, qualitative statements (“I am absolutely convinced”)
  • Relied on a novel computational measure of certainty in language: the Certainty Lexicon

Manipulation Pretests



  • Five pretests (N = 2,098) to obtain appropriately ordered stimuli
  • Participants rated the uncertainty expressed by several stimuli
  • Obtained four successful manipulations
    • Calibrated scale
    • Percentage statements
    • Two qualitative manipulations
  • Pretested with humans and, if applicable, Certainty Lexicon

Calibrated Scale



On a 1-9 scale (where 1 = highly uncertain and 9 = highly certain), …

Uncertainty Condition Statement
-4 My certainty level is a 9
-3 My certainty level is an 8
-2 My certainty level is a 7
-1 My certainty level is a 6
0 My certainty level is a 5
1 My certainty level is a 4
2 My certainty level is a 3
3 My certainty level is a 2
4 My certainty level is a 1

Calibrated Scale



On a 1-9 scale (where 1 = highly uncertain and 9 = highly certain), …

Uncertainty Condition Statement Median Rating Mean Rating
-4 My certainty level is a 9 -4 -3.26
-3 My certainty level is an 8 -3 -2.42
-2 My certainty level is a 7 -2 -1.67
-1 My certainty level is a 6 -1 -0.77
0 My certainty level is a 5 0 -0.10
1 My certainty level is a 4 1 1.08
2 My certainty level is a 3 2 1.76
3 My certainty level is a 2 3 2.79
4 My certainty level is a 1 4 2.55

Percentage Scale



Uncertainty Condition Statement
-4 I am 99% sure that
-3 I am 90% sure that
-2 I am 80% sure that
-1 I am 60% sure that
0 I am 50% sure that
1 I am 40% sure that
2 I am 20% sure that
3 I am 10% sure that
4 I am 1% sure that

Percentage Scale



Uncertainty Condition Statement Median Rating Mean Rating
-4 I am 99% sure that -4 -3.18
-3 I am 90% sure that -3 -2.47
-2 I am 80% sure that -2 -1.81
-1 I am 60% sure that -1 -0.65
0 I am 50% sure that 0 0.08
1 I am 40% sure that 1 0.97
2 I am 20% sure that 2 1.83
3 I am 10% sure that 3 2.24
4 I am 1% sure that 4 3.61

Qualitative Scale 1



Uncertainty Condition Statement
-4 I am absolutely convinced that
-3 I am convinced that
-2 I am reasonably convinced that
-1 I am partly convinced that
0 I am examining whether or not
1 I am reasonably unconvinced whether
2 I am unconvinced whether or not
3 I am desperately unconvinced whether
4 I’m genuinely unsure. I’m desperately unconvinced whether

Qualitative Scale 1



Uncertainty Condition Statement Median Rating Mean Rating
-4 I am absolutely convinced that -4 -3.30
-3 I am convinced that -3 -2.02
-2 I am reasonably convinced that -2 -1.36
-1 I am partly convinced that -1 -0.22
0 I am examining whether or not 0 0.22
1 I am reasonably unconvinced whether 1 0.44
2 I am unconvinced whether or not 2 1.35
3 I am desperately unconvinced whether 3 1.44
4 I’m genuinely unsure. I’m desperately unconvinced whether 4 3.02

Qualitative Scale 1



Uncertainty Condition Statement Median Rating Mean Rating Lexicon Score
-4 I am absolutely convinced that -4 -3.30 -2.95
-3 I am convinced that -3 -2.02 -2.38
-2 I am reasonably convinced that -2 -1.36 -1.55
-1 I am partly convinced that -1 -0.22 -0.80
0 I am examining whether or not 0 0.22 -0.17
1 I am reasonably unconvinced whether 1 0.44 0.37
2 I am unconvinced whether or not 2 1.35 1.08
3 I am desperately unconvinced whether 3 1.44 1.66
4 I’m genuinely unsure. I’m desperately unconvinced whether 4 3.02 2.88

Qualitative Scale 2



Uncertainty Condition Statement
-4 I am absolutely convinced that
-3 I am convinced that
-2 I am reasonably convinced that
-1 I am generally convinced that
0 I am examining whether
1 I am generally unconvinced whether
2 I am unconvinced whether or not
3 I am desperately unconvinced whether
4 I seriously don’t know… I’m desperately unconvinced whether

Qualitative Scale 2



Uncertainty Condition Statement Median Rating Mean Rating
-4 I am absolutely convinced that -4 -3.30
-3 I am convinced that -3 -2.02
-2 I am reasonably convinced that -2 -1.36
-1 I am generally convinced that -1 -1.18
0 I am examining whether 0 0.16
1 I am generally unconvinced whether 1 0.24
2 I am unconvinced whether or not 2 1.35
3 I am desperately unconvinced whether 3 1.44
4 I seriously don’t know… I’m desperately unconvinced whether 4 2.93

Qualitative Scale 2



Uncertainty Condition Statement Median Rating Mean Rating Lexicon Score
-4 I am absolutely convinced that -4 -3.30 -2.95
-3 I am convinced that -3 -2.02 -2.38
-2 I am reasonably convinced that -2 -1.36 -1.55
-1 I am generally convinced that -1 -1.18 -1.37
0 I am examining whether 0 0.16 -0.83
1 I am generally unconvinced whether 1 0.24 0.64
2 I am unconvinced whether or not 2 1.35 1.08
3 I am desperately unconvinced whether 3 1.44 1.66
4 I seriously don’t know… I’m desperately unconvinced whether 4 2.93 3.03

Manipulation Pretests

Experimental Design and Procedure



  • Five designs to confirm generalizability across different experimental tasks
  • Prolific participants randomly assigned to one of five experimental designs
    • Within each design, randomly assigned to one of nine experimental conditions
    • 6,000 participants; 1,200 per design; \(\pm\) 133 per condition
  • Three-round incentivized game
  • Paired with a leader who gives advice expressed in varying degrees of uncertainty
  • Advice quality and accuracy held constant
  • Measure attitudinal and behavioral consequences of leader-expressed uncertainty

Study 1: Decoding Task

Using the decoding scheme above, which of the following answers is correct?

A. YYMAM = LLAMA
B. YRAT = LENGTH
C. MNHUT = LAUGH
D. YVYL = LILY

Message from Team Leader:
I said D here. I’ve been asked to quantify for you how certain I am that it’s the right answer. On a 1-9 scale (where 1 = highly uncertain and 9 = highly certain), I’d say my certainty level is a(n) [1/2/3/4/5/6/7/8/9].

Study 2: Emotion Estimation

This face is expressing…
A. Pride
B. Contempt
C. Excitement
D. Anger

Message from Team Leader:
I picked Pride here. If I had to express my certainty in terms of a percentage, I’d say I am [1/10/20/40/50/60/80/90/99]% sure that I got this one right.

Study 3: Anagrams



Can you unscramble the following letters?

BIOGELO
A. YES: This set of scrambled letters has a valid solution
B. NO: This set of scrambled letters does not have a valid solution

Message from Team Leader:
I answered NO. I am…
Show qualitative uncertainty statement
  • genuinely unsure. I’m desperately unconvinced whether
  • desperately unconvinced whether
  • unconvinced whether or not
  • generally unconvinced whether or not
  • examining whether
  • generally convinced that
  • reasonably convinced that
  • convinced that
  • absolutely convinced that

…I’m right this time.

Study 4: Signaling game



  • Participants play P1 Type B
  • P2 does not know which type they are
  • Participants earn more if P2 plays right
  • Their goal is to signal to P2 that they are Type B

What is your selection?
A. 1
B. 2
C. 3
D. 4



Message from Team Leader:
I played 4. I am [1/10/20/40/50/60/80/90/99]% sure that’s the best choice.

Study 5: Prediction game



  • Participants report their own preferences
  • Participants predict preferences of 100 other survey-takers
  • Goal is to guess within 5 points of correct answer



How many of 100 people surveyed prefer having a cat to a dog?



Message from Team Leader:
I said 40 here. I’ve been asked to quantify for you how certain I am that it’s within 5 points of the right answer. On a 1-9 scale (where 1 = highly uncertain and 9 = highly certain), I’d say my certainty level is a(n) [1/2/3/4/5/6/7/8/9].

Attitudinal Outcomes



  • 7-point scales (1 = Strongly Disagree, 7 = Strongly Agree)
  • Leader effectiveness
    Show items
    • Is a good leader
    • Has strong leadership skills
    • Has the ability to lead other people
  • Warmth
    Show items
    • Friendly
    • Well-intentioned
    • Trustworthy
    • Warm
    • Good-natured
    • Sincere
  • Competence
    Show items
    • Competent
    • Capable
    • Independent
    • Intelligent
    • Skillful

Behavioral Outcomes



  • Advice-following/performance
    Show details In each round, we obtain a dichotomous measure indicating whether participants choose the leader’s recommended strategy (0 = no, 1 = yes). We sum the three separate indicators into a single measure of advice-following
  • Leader selection
    Show details At the end of 3 rounds, participants move on to a new incentivized game and are asked whether they would like to keep the same team leader or be paired with a new team leader (with similar performance). We obtain a single dichotomous measure indicating whether participants choose to keep the same leader (leader selection: 0 = no, 1 = yes)
  • Leader reward
    Show details Participants are asked how much, if anything, they would like to award the leader, on a bonus payment slider ranging from $0 to $5

Analysis plan



  • Assess linear effects of leader-expressed uncertainty (separately for each outcome)
    Show details
    • Five separate linear regression models
    • Random-effects meta-analysis to obtain pooled effect size estimate
    • Evidence for H1 (H2): Pooled estimate significant and negative (positive)
  • Assess nonlinear effects of leader-expressed uncertainty (separately for each outcome)
    Show details
    • Five separate two-lines tests
    • Random-effects meta-analysis to obtain pooled effect size estimate
    • Evidence for H3 if 1st line positive and 2nd line negative
  • Bayesian analysis if both tests are inconclusive for a particular outcome

Process

Who Shall Lead? The Effectiveness Implications of Self-Selection into Leadership Roles

Self-Selection into Candidate Pools



  • Leader selection processes rely on a strong default of self-selection
  • Individuals must actively “opt-in” to the candidate pool for leadership roles
  • Despite the ubiquity of this default…
  • Unknown how it affects composition and effectiveness of candidate pools for leadership roles

Effectiveness Implications



  • Examine the effectiveness implications of self-selection into candidate pools for leadership roles
  • Hypothesize that self-selection induces adverse leader selection, creating a candidate pool that is…
    • Underrepresented on attributes helpful to leadership effectiveness (e.g., prosociality)
    • Overrepresented on attributes harmful or unrelated to leadership effectiveness (e.g., ambition, overconfidence, narcissism)
  • Compare opt-in vs. opt-out selection mechanisms
  • Goal: identify institutional interventions that may…
    • Foster diversity in upper-level positions
    • Promote workplace advancement for individuals
      with strong leadership potential

Hypotheses



Opt-in (vs. opt-out) mechanisms…

H1: Decrease the aggregate effectiveness of candidate pools for leadership roles

H1a: Increase the prominence of effectiveness-inhibiting traits in candidate pools

H1b: Increase the prominence of effectiveness-irrelevant traits in candidate pools

H1c: Decrease the prominence of effectiveness-inducing traits in candidate pools